AO*A053  021 


UNCLASSIFIED 


•CNCRAL  RESEARCH  CORF  SANlA  BARBARA  CALIF  m/^ 

COST  REP0RTXN6  ELEMENTS  AND  ACTIVITY  COST  TRAOE-OFFS  FOR  OEFENS--ETC ( U ) 
MAY  77  C A BRAVER*  M M CARRIERE  Fl962a-76-C-01B0 


CR-l-7ti-V0L-2 


ESO*TR-77-tBt-VOL-2 


AD  No.- ^ ^ 

ODC  F1LE_CQ£^  ADA053021 


ESD-TR-77-262,  Vo\.  11 


COST  REPORTING  ELEMENTS  AND  ACTIVITY  COST 
TRADEOFFS  FOR  DEFENSE  SYSTEM  SOFTWARE 
(EXECUTIVE  SUMMARY) 


General  Research  Corporation 

P.  O.  Box  3587 

Santa  Barbara,  CA  93106 


May  1977 


Approved  for  Public  Release; 
Distribution  Unlimited. 


i 

i 


i 

I 


I 


Prepared  for 

DEPUTY  FOR  COMMAND  AND  MANAGEMENT  SYSTEMS 
ELECTRONIC  SYSTEMS  DIVISION 
HANSCOM  AIR  FORCE  BASE,  MA  01731 


I 


% 


LEGAL  NOTICE 

When  U.  S.  Government-  drawings,  specifications  or  other  data  are  used  for  any 
purpose  other  than  a definitely  related  government  procurement  operotion,  the 
government  thereby  incurs  no  responsibility  nor  any  obligation  whatsoever;  and 
the  fact  that  the  government  may  have  formulated,  furnished,  or  in  any  way  sup- 
plied the  said  drawings,  specifications,  or  other  data  is  not  to  be  regarded  by 
implication  or  otherwise  as  in  any  manner  licensing  the  holder  or  any  other  person 
or  conveying  any  rights  or  permission  to  manufacture,  use,  or  sell  any  patented 
invention  that  may  in  any  way  be  related  thereto. 


OTHER  NOTICES 


Do  not  return  this  copy.  Retain  or  destroy. 

This  technical  report  has  been  reviewed  and  is  approved  for  publication. 


WILLIAM  J.  WHITE,  Captain,  USAP 
Project  Engineer 


FOR  THE  COMMANDER 


JOHN  T.  HOLLAND,  Lt  Col,  DSAP 
Chief,  Techniques  Engineering  Division 


TORU  YAMAMOTO,  Colonel,  USAP 
Director,  Computer  Systems  Engineering 
Deputy  for  Command  & Management  Systems 


SeCUWlTY  CLASSIFICATION  OF  THIS  l»ACE  QWlol  Pl«  gnCf O 

I (li)  REPORT  DOCUMENTATION  PACE^ 


wcpoywuHBtw  r a.  oovt  acccuion  no. 

i ^\^R-77-262ry^r1t^  ^ 

Cost  Reporting  Elements  and  Activity  Cost  Trade-/ 
offs  for  Defense  System  Software  « j ( 

Volume^  II.  w )'  ; 

~ AUTMQWC.)  ^ 

C.A. /graver,  W.M. /^arriere,  j 
I*’  E.E,/Balkovich^R.^hibodeau  j * 

»■  PER70NMING  ORGANIZATION  NAME  AND  AOOMCSS 

General  Research  Corporation 

P.O.  Box  3587 

Santa  Barbara,  CA  93105 

II.  CONTROLLING  OFFICE  NAME  ANO  ADDRESS 

Electronic  Systems  Division  i 

Air  Force  Systems  Command 
Hanscom  AFB,  MA  01731 

14.  monitoring  agency  name  S AODRESSCI/ tfff.rant  from  Cantnllhtg  OWe») 


16.  distribution  statement  (oI  thit  Hapotl) 

Approved  for  Public  Release;  Distribution  Unlimited. 


{ '7.  DISTRIBUTION  STATEMENT  (el  the  mbelreel  entered  In  Sloe*  20,  II  ditfenni  Item  Me^orl) 


IS.  SUPPLEMENTARY  NOTES 

ESD  Project  Engineer:  Captain  William  J.  White  (MCI) 


1 1*.  KEY  WORDS  (Cenllnum  on  teretee  mide  II  neeeteery  end  Idenlllr  *F  Meek  nimker) 


READ  mSTRUCTtONS 
BEFORE  COMPLETING  FORM 
S.  RECIPIENT’S  CATALOG  NUMBER 

(!*^T~Mar  76  ~ H Mar  77t  ^ 

S.  T'^E  OF  RSPORTlNtEmOO  COVtREO 

9T  Final 

o^erformiwo  pro,  report  number 

Id  ICR-l-721-VQl-«d?l  ^ 
^nNVMAef  <?x’jiM^J8iiiiir«j — 


F19628-76-C 


program  ELCMCNT.  project.  TAt 

AREA  a WORK  UNIT  NUSni^RS 

PE63101F,  Proj  ^4/^34 


56<  aJl  I 

IS.  SECURITY  class,  (el  Ale  tfeetl) 

Unclassified 

^Sa.  oeclassification7oownoradino 
SCNEDULE 

' N/A 


¥ 


Software  Size 
Computer  Program 
Components 


Software  Software  Life  Cycle  Software  Size 

Costs  Software  Development  Computer  Program 

Reporting  System  Software  Maintenance  Components 

Software  Costs  Software  Manhour  Estimates 

Software  Reporting  System  Software  Product  Description 

SO.  abstract  fCofitImM  on  rovoroo  olcto  If  noeoooorr  Sp  Moek  iwSor) 

^Thls  technical  report  examines  the  costs  of  developing  and  maintaining  computer 
software  for  major  defense  systems.  A process  model  is  described  which  depicts 
the  relations  among  actlvites  and  phases  of  the  software  life  cycle,  identifies 
the  product  and  cost  information  that  is  normally  available,  and  specifies  the 
milestones.  This  process  model  is  normally  used  as  the  basis  for  selecting  the 
elements  of  a software  cost  reporting  system.  The  suggested  reporting  system 
also  Includes  descriptions  of  the  final  product,  time  phasing  of  product  develop' 
ment,  a standardized  list  of  Computer  Program  Components,  and  a atandard< zgd  ^ 


DO  1473 


edition  of  I NOV  61  It  OBtOLETE 


security  classification  of  this  base  rWSM  DMe  BNIWmO 


7 y 


security  classification  of  this  PAGEflWi»n  Dmim  Bnlmnq) 


f 

(Block  20  continued) 
list  o£  labor  categories. 

During  the  study,  data  was  collected  from  several  sources  including  the 
following  Air  Force  organizations; 

Electronic  Systems  Division 
Aeronautical  Systems  Division 
Space  and  Missile  Systems  Organization  x 
Data  Systems  Design  Center 
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trade-offs  in  cost  between  phases  is  demonstrated.  The  report  also  contalr 
estimating  relationships  for  evaluating  the  cost  effects  of  software  size, 
computer  capacity  constraints,  programming  language,  and  changes  in 
requirements.  It  also  addresses  the  separation  of  two  activities,  error 
correction  and  product  improvement,  during  the  maintenance  phase  of  the 
life  cycle.  Results  are  Integrated  with  other  software  cost  estimating 
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ABSTRACT 


This  technical  report  examines  the  costs  of  developing  and  main- 
taining computer  software  for  major  defense  systems.  A process  model  is 
described  which  depicts  the  relations  among  activities  and  phases  of  the 
software  life  cycle.  Identifies  the  product  and  cost  Information  that  is 
normally  available,  and  specifies  the  milestones.  This  process  model  Is 
used  as  the  basis  for  selecting  the  elements  of  a software  cost  report- 
ing system.  The  suggested  reporting  system  also  Includes  descriptions 
of  the  final  product,  time  phasing  of  product  development,  a standardized 
list  of  Computer  Program  Components,  and  a standardized  list  of  labor 
categories. 

During  the  study,  data  was  collected  from  several  sources  includ- 
ing the  following  Air  Force  organizations: 

Electronic  Systems  Division 
Aeronautical  Systems  Division 
Space  and  Missile  Systems  Organization 
Data  Systems  Design  Center 

Cost  estimating  relationships  for  each  phase  of  the  software  life 
cycle  are  explored,  using  the  process  model  and  the  data.  The  Importance 
of  trade-offs  in  cost  between  phases  is  demonstrated.  The  report  also 
contains  estimating  relationships  for  evaluating  the  cost  effects  of  soft- 
ware size,  computer  capacity  constraints,  progrannlng  language,  and  changes 
in  requirements.  It  also  addresses  the  separation  of  two  activities,  error 
correction  and  product  improvement,  during  the  maintenance  phase  of  the 
life  cycle. 

Results  are  reported  In  Vol.  1 and  suoanarlzed  In  this  volume.  Section 
1 Is  a 17-page  summary  of  the  entire  project.  Sections  2 and  3 highlight 
the  reporting- system  and  estlmatlng-relatlonshlp  results  respectively. 
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SUMMARY 


1.1  GOALS  ^ 

In  April  1976,  General  Research  Corporation  (GRC)  began  a study  of 
"Life-Cycle  Costing  of  Major  Defense  System  Software  and  Computer  Resources," 
Contract  F19628-76-C-0180.  The  purpose  was  to  assist  Air  Force  Program 
Offices  and  staff  agencies  in  estimating,  reporting,  and  controlling  the 
cost  of  the  computer  software  embedded  in  defense  systems.  The  study  was 
performed  under  the  direction  of  the  Electronic  Systems  Division  (AFSC) , 
Computer  Systems  Engineering  Office  (TOI) . Captain  William  White  was  the 
Project  Officer  and  coordinator  of  the  data-collection  survey,  and  has 
provided  many  helpful  comments  during  the  course  of  this  study. 

The  study  had  two  goals: 

1.  To  define  the  elements  of  a system  for  reporting  the  costs  of 
developing  and  maintaining  computer  software 

2.  To  develop  resource  estimating  relationships  for  the  different 
phases  of  the  software  life  cycle 

The  reporting  system  has  two  purposes:  (1)  collection  of  consistent 
and  accurate  data,  to  be  used  for  estimating  the  costs  of  future  software, 
and  (2)  provision  of  timely  and  relevant  Information  concerning  the  pro- 
gress of  software  development,  to  be  used  by  Program  Offices  in  cost  con- 
trol. It  is  truly  unfortunate  that  the  need  for  a reporting  system  still 
exists.  As  early  as  1966,  reporting  systems  were  being  devised  for  collect- 
ing data  on  software  cost.^  So  far,  however,  no  uniform  reporting  system 
has  been  adopted  by  the  Air  Force  or  DoD. 

The  second  goal,  development  of  resource  estimating  relationships 
for  the  different  phases  of  the  software  life  cycle,  has  also  had  a long 
history  of  attempts.  In  general,  previous  studies  concentrated  on  esti- 
mating total  cost.  Their  lack  of  success  was  not  due  to  lack  of  compe- 
tence; many  imaginative  relationships  were  developed  and  tested. 
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Unfortunately,  variances  remained  too  large  for  such  relationships  to  be 
useful  for  estimation,  as  witnessed  by  the  fact  that  authors  did  not 
recommend  the  use  of  their  findings  to  estimate  software  cost. 

In  an  attempt  to  attack  the  problem  from  a different  perspective, 
we  were  directed  to  develop  resource  estimating  relationships  (that  Is, 
equations  for  man-hours,  computer-hours,  and  elapsed  time)  for  each  phase 
of  the  software  life  cycle  separately,  rather  than  for  total  man-hours  or 
costs.  The  relationships  were  to  be  based  upon  currently  available  data 
and  developed  using  parametric  cost  estimating  techniques. 

The  following  principles  evolved  during  the  course  of  the  study  and 
guided  our  progress  towards  the  study's  goals. 

First,  both  the  recommended  cost  reporting  system  and  the  estimating 

relationships  must  be  related  to  the  Air  Force  procurement, process. 

Attempts  now  underway  to  standardize  this  process  are  reported  In  AFR 
2 

800-14,  which  we  have  used  as  our  guide  In  developing  a Process  Model, 
tempered  by  visits  to  Program  Offices  and  by  our  own  experience. 

Second,  the  cost  reporting  system  must  meet  two  objectives.  It 
must  provide  Information  which  will  help  the  Program  Offices  to  control 
software  costs.  At  the  same  time  It  must  be  used  to  compile  data  on  many 
systems,  using  standard  definitions,  so  that  better  cost  estimating  rela- 
tionships can  eventually  be  developed. 

Third,  resource  estimating  relationships  for  man-hours,  computer- 
hours,  and  elapsed  time  should  be  developed  separately  for  each  phase  of 
the  software  life  cycle,  using  parametric  cost  estimating  techniques.  The 
relationships  should  estimate  resources  as  functions  of  Inputs  normally 
available  to  the  Air  Force.  A relationship  which  depends  on  Input  data 
not  available  to  the  Program  Office  (for  example,  competence  of  the  Indi- 
vidual programmer)  might  be  statistically  valid  but  would  be  useless  In 
practice. 


The  first  goal  of  the  study  has  been  achieved.  The  elements  of  a 
reporting  system  have  been  defined.  Elements  to  describe  product  status 
as  well  as  resource  consumption  are  Included. 

The  development  of  estimating  relationships  was  less  s'accessful. 

New  estimating  relationships  were  developed,  but  they  do  not  constitute 
a complete  model  of  software  cost,  nor  a complete  set  of  resource  esti- 
mating relationships  for  each  phase  of  the  life  cycle.  They  can  only  be 
viewed  as  first  steps  in  developing  such  a model. 

The  reasons  for  this  partial  achievement  are  described  in  Sec.  1.2, 
Obstacles.  Our  conclusions  are  presented  in  Sec.  1.3,  and  recommendations 
in  Sec.  1.4.  Section  1 constitutes  a complete  overview  of  the  study. 

More  details  concerning  the  reporting  system  and  the  estimating 
relationships  are  given  in  Secs.  2 and  3,  respectively. 


Volume  1 also  Includes  (in  Sec.  7)  a description  of  the  state  of  the 
art  in  software  cost  estimating  techniques,  which  Incorporates  the  results 
of  our  study  with  those  of  others  reported  in  the  literature.  The  techni- 
ques still  have  wide  variances,  and  caution  should  be  used  in  applying  them. 
This  portion  of  Vol.  1 is  not  summarized  in  this  volume. 

1 . 2 OBSTACLES 

The  obstacles  we  met  included  definitional  problems,  data  problems, 
and  conceptual  problems.  Each  is  discussed  below. 

1.2.1  Definitional  Problems 

One  of  our  first  tasks  was  to  develop  a model  of  the  software  life 

2 

cycle.  The  basis  for  the  model  was  AFR  800-14,  which  divides  the  soft- 
ware life  cycle  into  six  phases;  (1)  analysis,  (2)  design,  (3)  coding 
and  checkout,  (4)  test  and  integration,  (5)  installation,  and  (6)  opera- 
tions and  support.  The  first  four  phases  are  generally  referred  to  as 
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the  development  period.  The  process  is  a complicated  one.  Including 
feedback  and  overlapping  among  the  phases. 

A further  complication  Is  the  dual  use  of  the  terms  for  life-cycle 
phases.  For  example,  "Design"  refers  to  a set  of  activities:  functional 
allocation  of  the  software  to  modules,  development  of  the  detailed  design 
as  represented  by  flow  diagrams,  etc.  On  the  other  hand,  "Design"  Is  also 
commonly  described  as  beginning  after  the  Preliminary  Design  Review  (PDR) 
and  ending  at  the  Critical  Design  Review  (CDR) . Thus,  both  an  "activity" 
and  a "milestone"  use  of  the  term  are  common. 

This  ambiguity  caused  considerable  problems  during  the  study.  One 
consequence  was  Inconsistencies  In  the  data.  Some  sources  reported  data 
on  an  activity  basis,  but  most  sources  derived  their  reports  on  resource 
consumption  from  time-phased  expenditure  records  which  can,  at  best,  be 
associated  with  milestones. 

Although  we  feel  that  the  dual  use  of  these  terms  should  be  elimi- 
nated, the  problem  Is  so  widespread  that  redefining  the  terms  In  this 
study  would  probably  have  simply  caused  more  confusion.  We  have  chosen, 
therefore,  to  specifically  Identify  the  problem  and  to  show  how  the  two 
uses  are  related.  (A  mapping  of  one  use  onto  the  other  Is  contained  In 
Table  2.3).  In  addition,  we  define  the  dual  uses  of  the  terms  In  Sec.  2 
of  Vol.  1 and  adopt  the  following  convention:  "When  the  terms  analysis. 
design,  and  so  on  are  used  without  qualifiers,  an  activity  definition  Is 
intended.  When  the  terms  are  followed  by  phase  or  milestone . a milestone 
definition  Is  Intended." 

1.2.2  Data  Problems 

As  In  previous  studies  of  software  cost,  the  location  and  collec- 
tion of  accurate  and  consistent  data  was  a problem.  We  first  attempted 
to  collect  data  from  seven  representative  Program  Offices  In  the  Air 
Force  Systems  Coinnand:  three  at  the  Electronic  Systems  Division  (ESD) , 
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two  at  the  Aeronautical  Systems  Division  (ASD) , and  two  at  the  Space  and 
sile  Systems  Organization  (SAMSO) . 

The  results  of  our  visits  to  these  Program  Offices  were  extremely 
important  in  shaping  the  study's  direction,  especially  in  defining  the 
elements  of  the  recommended  reporting  system.  The  Program  Offices  have 
a difficult  problem  in  control  Ing  the  cost  of  software.  A major  reason 
is  that  costs  are  reported  at  far  too  aggregated  a level — in  some  cases 
as  only  a single  total.  The  problem  is  complicated  by  the  Inability  to 
relate  software  cost  to  progress  towards  the  completion  of  software  deve- 
lopment. Progress  is  often  reported  by  the  percentage  of  estimated  man- 
hours expended  to  date;  a less  than  reliable  measure  of  the  amount  of 
software  actually  completed.  Also,  since  there  is  no  standard  list  of 
software  products  (end  items)  for  which  costs  have  been  collected  in 
previous  projects,  the  Program  Offices  have  few  precedents  upon  which  to 
base  cost  estimates. 

Recorded  cost  data  exhibits  a great  deal  of  variability  between 
developments.  This  is  due  not  only  to  the  size  and  complexity  of  the 
developments,  but  also  to  the  following: 

1.  Fixed-price  contracts  have  gone  to  ceiling.  Hence  the  costs 
reported  understate  the  costs  incurred. 

2.  Cost-reimbursement  contracts  have  been  augmented  by  supple- 
mentary  agreements  that  incorporate  changes  in  requirements. 
This  has  resulted  in  redoing  work  already  completed,  so  that 
costs  are  abnormally  high  per  line  of  code  delivered. 

3.  Level-of-ef fort  maintenance  contracts  Include  costs  for  pro- 
duct improvement  as  well  as  error  correction.  Costs  are  dif- 
ficult to  assign  to  these  two  different  functions. 


it 
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Software  costs  are  often  hidden  In  hardware  development  due 
to  the  difficulty  In  some  Instances  of  allocating  the  costs 
between  hardware  and  software  (e.g.,  In  systems-englneering 
functions) . 

For  these  reasons,  we  concluded  that  the  Program  Offices  were  not 
a good  source  of  data  on  the  resource  requirements  for  Individual  life- 
cycle  phases,  within  the  study's  time  and  cost  constraints.  The  quality 
of  the  data  was  poor  because  of  the  lack  of  uniform  reporting,  as  described 
above.  Also,  there  was  not  sufficient  detail  relating  the  degree  of  pro- 
duct completion  to  cost.  Finally,  what  data  existed  was  difficult  to  col- 
lect and  did  not  include  all  the  life-cycle  phases,  especially  mainte- 
nance. At  some  point  the  data  from  the  Program  Offices  should  be  com- 
piled Into  a usable  data  base,  but  the  effort  required  will  be  well  beyond 
the  resources  of  this  contract. 

At  the  first  Technical  Direction  meeting,  we  proposed  to  take  the 
following  study  approach.  Data  would  be  collected  from  readily  available 
sources  In  which  data  on  software  product  development  could  be  measured. 

Of  special  concern  was  visibility  into  both  development  and  maintenance 
portions  of  the  life  cycle.  SAMSO  contractors  appeared  to  be  a good 
source  because  they  had  both  development  and  maintenance  responsibility. 

In  particular,  we  gathered  data  from  the  SAMSO  Program  Office  responsible 
for  the  development  of  general-purpose  support  software  for  the  Satellite 
Control  Facility  (SCF) . 

A major  SCF  subsystem  Is  the  Advanced  Orbital  Ephemerls  Subsystem 
(AOES) , which  contains  more  than  four  hundred  programs  Including  compilers, 
operating  systems,  data  reduction  and  presentation  utilities,  and  orbit 
planning  and  analysis  utilities.  The  AOES  is  maintained  by  two  associated 
contractors  and  an  Integration  contractor.  To  achieve  configuration  con- 
trol, the  integration  contractor  maintains  a data  base  of  the  characteris- 
tics of  the  AOES  programs,  and  records  describing  their  modifications. 
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Unfortunately,  this  data  cannot  be  correlated  with  the  costs  of  the  compo- 
nent activities.  Also  data  on  the  size  of  the  programs  is  missing.  How- 
ever, it  still  was  possible  to  use  this  data  to  investigate  the  properties 
of  the  operations  and  support  phase  of  the  software  life  cycle. 

The  data  we  collected  describes  four  successive  revisions  or  "re- 
leases" of  the  AOES,  and  the  maintenance  histories  of  those  releases  once 
they  became  operational.  For  each  program  of  each  release,  we  collected 
(1)  the  size  as  delivered,  (2)  the  number  of  problems  reported  and  re- 
solved, (3)  the  size  of  the  Part  II  Specifications,  (4)  the  design  changes 
Incorporated  with  that  release,  and  (5)  the  relationship  of  system  inte- 
gration tests  to  the  design  changes.  In  addition,  we  collected  data  on 
the  man-months  of  development  and  the  man-months  of  maintenance  effort, 
and  the  schedules  for  developing,  testing,  and  operating  the  releases. 

A third  data  source  was  located  late  in  the  study  at  the  Air  Force 
Data  Systems  Design  Center  (AFDSDC) . Software  development  projects  at 
the  Design  Center  are  supported  by  an  automated  system  called  the  Planning 
and  Resource  Management  Information  System  (PARMIS) . The  system  is  a 
project  planning  and  management  aid  that  enables  project  managers  to  enter 
estimated  schedules  and  manpower  allocations  and  to  receive  regular 
reports  on  actual  utilization. 

The  histories  of  over  2,000  software  developments  are  stored  on  the 
system.  PARMIS  files  its  data  by  activity;  a project  manager  entering  a 
new  project  into  the  system  estimates  start  and  end  times  and  manhours 
for  each  activity,  selected  from  a standard  set  of  activities  contained 
in  the  PARMIS  Catalog.  The  Catalog's  activity  codes  and  activity  group 
codes  make  it  possible  to  allocate  the  costs  of  the  development  activi- 
ties among  the  phases. 

PARMIS  records  planned  and  actual  man-hours,  start  dates,  and  end 
dates  (for  each  activity)  under  four  Job  classifications:  Functional 
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Analyst,  Data  Systems .Analyst,  Programmer,  and  Support  Personnel.  The 
project  manager  Is  free  to  revise  his  estimated  dates  and  man-hours,  but 
the  system  always  maintains  and  reports  the  estimates  made  when  the  pro- 
ject was  entered  Into  the  system. 

PARMIS  has  two  drawbacks.  First,  It  contains  little  Information  on 
the  nature  of  the  software  being  developed.  We  had  to  obtain  Information 
on  the  size  and  functions  of  programs  by  personal  Interview.  Although 
people  at  the  Design  Center  were  very  cooperative,  the  Interviews  made 
data  collection  a lengthy  process.  Second,  PARMIS  contains  only  business- 
application  computer  programs  (payroll,  administration,  etc.).  We  do  not 
consider  this  a serious  limitation,  however,  as  the  development  process 
Is  similar  to  that  of  major  defense-system  software,  and  therefore  a great 
deal  of  Information  can  be  gained  from  studying  the  development  process 
at  AFDSDC. 

We  also  used  some  data  on  our  own  software  developments,  and  data 
from  previous  studies  reported  In  the  open  literature.  However,  most  of 
this  data  was  too  aggregated  to  be  used  In  phase-specific  studies. 

In  short,  collection  of  software  cost  data  continues  to  be  a prob- 
lem (highlighting  the  need  for  a good  reporting  system) . Of  the  data 
bases  Investigated,  PARMIS  was  the  most  complete,  containing  records  of 
expenditures  related  to  software  development  activities  and  phases.  In 
addition,  the  AFDSDC  retains  responsibility  for  software  maintenance,  and 
therefore  also  collects  data  on  errors.  Our  study  represented  only  a 
limited  attempt  to  tap  this  data  base,  examining  only  20  out  of  2,000 
recorded  developments.  Of  these  20,  we  were  only  successful  In  obtain- 
ing complete  data  on  seven,  due  to  Incomplete  questionnaires.  Thus,  the 
limited  use  of  PARMIS  In  this  study  should  be  considered  a beginning  of 
Its  Investigation  and  not  an  end. 


1.2.3  Conceptual  Problems 

The  last  major  obstacle  concerned  the  attempt  to  develop  resource 
estimates  for  each  of  the  life-cycle  phases  separately.  While  we  were 
collecting  data,  we  hypothesized  resource  estimating  relationships  and 
causal  parameters  for  each  of  the  life-cycle  phases.  Section  3 of  Vol.  I 
documents  the  results. 

In  trying  to  calibrate  these  relationships  with  the  PARMIS  data, 
it  became  very  clear  that  the  phases  are  not  Independent.  That  is,  the 
resource  consumption  during  Test  and  Integration  depends  on  the  amount 
of  time  spent  on  Analysis  and  Design;  the  number  of  errors  found  during 
Maintenance  Is  dependent  upon  the  amount  and  quality  of  Development  Test- 
ing; and  so  on. 

Although  conceptually  a model  Incorporating  these  trade-offs  Is 
more  difficult,  it  does  offer  the  opportunity  to  provide  more  accurate 
estimating  relationships  and  to  discover  optimal  resource  allocations 
among  phases.  Figure  1.1  shows  such  a trade-off  relationship  for  five 
PASMIS  data  points.  Although  five  points  Is  far  too  small  a sample  to 
offer  conclusive  results  (and  other  relationships  could  fit  the  data 
equally  well),  the  possibility  of  discovering  these  types  of  trade-offs 
Is  exciting,  and  PARMIS  offers  a good  vehicle  for  the  attempt. 

These  trade-offs  may  hold  the  key  for  determining  why  aggregated 
estimating  techniques  have  such  wide  variance.  Data  used  to  derive  these 
aggregated  relationships  Included  less  costly  developments,  with  efficient 
resource  allocations  among  the  phases,  along  with  more  costly  develop- 
ments with  Inefficient  allocations.  Also,  discovering  efficient  alloca- 
tions will  give  the  Program  Offices  Insight  Into  the  planning  of  soft- 
ware development.  Preliminary  investigations  along  these  lines  are 
described  In  Secs.  4 and  5 of  Vol.  I. 
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Figure  1.1.  Relationship  Between  Analysis  and  Design  Time  and 
Programming  and  Testing  Time  (Volume  I,  Fig.  5.10) 


1.3  CONCLUSIONS 

Although  these  obstacles  caused  considerable  problems  In  meeting 
the  objectives  of  the  study,  we  did  complete  the  selection  of  elements 
for  the  cost  reporting  system  and  develop  some  new  resource  estimating 
relationships.  Significant  findings  are  reported  In  this  section,  organ- 
ized around  the  two  study  goals. 

1.3.1  Reporting-System  Elements 

Before  summarizing  our  findings,  we  briefly  describe  the  philosophy 
used  to  pick  the  specific  elements  and  categories  of  data  to  be  reported. 

One  approach  would  be  to  define  the  data  that  is  wanted  and  develop  the 
reporting  system  necessary  to  collect  It,  assuming  that  the  data  Is  there  • 

for  the  asking  and  that  once  defined,  all  that  Is  required  Is  the  anney 
to  finance  Its  collection.  In  effect,  many  of  the  previously  proposed 
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data  collection  systems  take  this  point  of  view.  An  Important  question 
to  resolve  Is  why  they  were  not  Implemented. 

One  possible  explanation  Is  that  the  cost  of  collecting  data  for 
these  systems  was  simply  too  high.  However,  we  feel  that  an  equally  sig- 
nificant reason  Is  contractors'  unwillingness  to  provide  data  In  as  much 
detail  as  the  proposed  systems  called  for.  In  effect,  what  a contractor 
Is  technically  able  and  willing  to  report  Internally  for  management  con- 
trol Is  far  more  than  what  he  Is  willing  to  share  with  the  government. 
Excessive  requirements,  therefore,  will  meet  with  stiff  resistance  from 
contractors  or  even  discourage  them  from  continuing  to  provide  the  ser- 
vices; neither  of  which  Is  desirable. 

In  selecting  the  data  to  be  collected,  we  have  tried  to  be  sensi- 
tive to  this  problem.  In  general,  we  have  recommended  more  detail  In 
reports  that  are  only  one-time  requirements.  We  have  recommended  less  In 
regularly  required  reports,  but  still  sufficient  for  control  by  Program 
Offices  and  the  development  of  estimating  relationships. 

Section  8 of  Vol.  I presents  our  recommendations  for  resource-data 
elements.  They  call  for  the  regular  collection  of  data  on  costs,  man- 
hours, computer-hours,  and  the  degree  of  product  completion.  Other  In- 
formation to  be  collected  at  specified  milestones  Includes  a detailed 
description  of  the  software  product,  the  characteristics  of  the  computer 
system  used  to  develop  and/or  maintain  the  software,  etc.  This  Informa- 
tion Is  much  more  detailed  than  that  required  to  be  regularly  reported; 
however,  the  Infrequency  of  reporting  should  ease  the  burden  on  the 
contractor. 

A summary  of  the  reporting  system  elements  Is  contained  In  Sec.  2 
of  this  volume.  Important  Innovations  Include  the  following: 
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L.  A moving  milestone  (source-code  baseline)  providing  a contin- 
uous measurement  of  progress  towards  completion.  This  mile- 
stone separates  Coding  and  Checkout  from  Test  and  Integra- 
tion. Each  time  a source  deck  is  completed  (coded  and  debug- 
ged) , the  deck  is  "baselined"  and  future  work  on  that  part 
of  the  program  will  be  recorded  against  Test  and  Integration. 
Thus,  work  in  these  two  phases  will  go  on  concurrently  and 
data  will  be  continuously  available  on  the  percentage  of  the 
product  which  has  finished  the  initial  coding  and  debugging 
phase  and  is  in  the  Test  and  Integration  phase. 

2.  No  requirement  to  separate  redesign  from  recoding  efforts  in 
the  Test  and  Integration  phase.  We  speculate  that  this 
requirement  in  earlier  cost  reporting  systems  has  been  one 
of  the  reasons  they  were  not  Implemented.  Redesign  and  re- 
coding efforts  in  this  period  are  often  done  by  the  same  per- 
son and  therefore  difficult  to  separate.  If  separate  report- 
ing were  required,  it  would  probably  be  artificial,  so  the 
information  content  would  be  suspect. 

* 

3.  A preliminary  standardized  list  of  end  items  (CPCs  ) for  com- 
paring resource  consumption  among  software  developments.  This 
list  will  serve  to  disaggregate  the  software  product  into 
meaningful  parts.  Ultimately,  software  will  be  described  by 
its  parts,  with  estimates  made  at  the  parts  level  and  then 
aggregated,  much  as  hardware  estimates  are  made  today.  The 
list  is  preliminary  in  that  we  are  confident  that  additions 
will  be  made  upon  review  by  the  software  community. 

4.  Separation  of  maintenance  into  error  response  and  product 
improvement . Only  the  need  for  error  response  should  be 
associated  with  the  original  software  development  project. 
Product  Improvement  (through  ECPs)  should  undergo  separate 


Computer  Program  Components  (CPCs)  are  in  general  a disaggregation  of 
Computer  Program  Configuration  Items  (CPCIs) . 


I 
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approval  and  be  subject  to  data  gathering  as  another  develop- 
ment project. 

5.  Identification  of  differences  In  ''milestone"  and  “activity” 
definitions  for  Analysis,  Design,  Coding,  and  Testing.  The 
Inconsistent  use  of  these  terms  has  been  a source  of  confusion 
during  the  entire  effort,  as  previously  discussed,  and  has 
undoubtedly  contributed  to  confusion  In  other  attempts  at 
data  gathering  and  analysis.  Ultimately,  different  terms  for 
the  milestone  and  activity  uses  should  be  selected. 

In  addition  to  defining  the  data  elements,  we  have  demonstrated  one 
way  of  mapping  these  elements  into  an  already  existing  Air  Force  reporting 
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system  (Program  Breakdown  Codes — AFSCM  173-4).  Hence,  the  mechanics  of 
data  retrieval  will  not  be  severe. 

We  believe  these  innovations  will  Improve  the  chances  that  contrac- 
tors will  accept  a reporting  system,  without  sacrificing  the  needs  of  the 
Program  Offices  for  cost  control  data  and  the  long-term  needs  of  the  Air 
Force  for  better  data  to  Improve  cost  estimation. 

1.3.2  Software  Cost  Estimating  Relationships 

Our  second  major  objective  was  to  develop  Improved  software  cost 
estimating  relationships  (CERs) . A significant  amount  of  work  had  been 
previously  devoted  to  this  task  by  competent  groups,  focusing  on  estimat- 
ing total  man-hours  or  costs.  Results  had  been  disappointing,  with 
derived  relationships  exhibiting  large  variances. 

As  a result,  we  were  directed  to  take  a new  approach  and  focus  our 
attention  on  the  development  of  separate  estimating  relationships  for  each 
life-cycle  phase.  Our  efforts  have  been  concentrated  on  man-hour 
estimation. 


13 


During  the  contract  we  first  developed  a process  nodel  to  describe 
the  relations  between  activities  and  phases  and  to  Identify  data  sources 
normally  available  during  the  life  cycle.  This  model  Is  described  In  de- 
tail in  Sec.  2 of  Vol.  1.  We  then  hypothesized  estimating  relationships 
for  each  of  the  life-cycle  phases. 

An  important  factor  In  all  the  estimating  relationships  was  the 
size  of  the  software.  Various  measures  of  size  have  been  used  In  earlier 
studies,  and  there  has  been  a great  deal  of  debate  on  whether  a measure 
of  source  code  or  object  code  Is  the  most  appropriate.  In  estimating 
cost  separately  for  each  phase  of  the  life  cycle,  much  of  this  debate 
can  be  sidestepped  by  choosing  object  code  as  the  measure  In  some  phases 
and  source  code  In  others.  Our  choice  Is  to  use  object  code  as  the  mea- 
sure In  all  phases  except  Coding  and  Checkout.  In  that  phase,  source 
code  is  the  product  being  manufactured  and  seems  to  us  the  most  appro- 
priate measure  of  the  effort. 

Beyond  this  choice,  there  Is  further  debate  on  what  to  count  In 
measuring  the  size  of  software;  specifically,  whether  to  Include: 

Throwaway  code — code  written  and  used  for  the  development  of  the 
delivered  programs,  but  not  Itself  delivered 

Converted  code — code  taken  from  earlier  programs  but  adapted  In 
detail  for  this  development 

Off-the-shelf  code — code  taken  from  earlier  programs  without 
adaptation 

Computer  data — code  that  supplies  values  for  variables  in  the  pro- 
gram (such  as  DATA  statements  In  FORTRAN  programs) 

Comment  statements — code  Intended  to  be  read  by  people  and  not  by 
the  computer 

Our  choices  are  as  follows.  For  all  phases  but  Coding  and  Checkout,  the 
measure  of  size  Is  the  number  of  object-code  Instructions  delivered. 
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excluding  only  computer  data  (and  of  course  throwaway  code,  which  is  not 
delivered).  In  the  Coding  and  Checkout  phase,  our  recommended  measure 
Is  the  number  of  source-code  Instructions  delivered,  again  excluding 
throwaway  code  and  computer  data,  and  also  excluding  off-the-shelf  code. 
These  choices  are  explained  In  Sec.  3 of  Vol.  I. 

We  then  attempted  to  fit  the  PARMIS  data  to  the  hypothesized  rela- 
tionships. Important  findings  Include: 

1.  Accurate  estimating  relationships  for  each  life-cycle  phase 
cannot  be  developed  Independent  of  the  other  phases.  The 
data  when  plotted  exhibit  significant  variation.  As  a con- 
sequence, an  estimate  of  total  man-hours  by  estimating  each 
life-cycle  phase  and  then  adding  will  have  less  precision 
than  an  estimate  made  on  the  total. 

2.  Estimating  the  trade-offs  between  the  life-cycle  phases  Is 
of  prime  Importance.  Only  then  will  the  variation  in  the 
total  man-hour  estimates  be  reduced,  l.e.,  the  accuracy  of 
the  estimating  relationship  be  Increased.  In  effect,  covari- 
ances must  be  Incorporated  In  the  estimators.  Reducing  the 
risk  In  software  cost  estimates  Is  dependent  on  quantifying 
these  relations  between  the  life-cycle  phases. 

3.  Estimating  the  trade-offs  can  also  lead  to  the  development  of 
rules  for  optimal  allocation  of  resources  among  life-cycle 
phases . Application  of  the  rules.  In  turn,  will  lead  to  less 
costly  or  lower-risk  developments.  In  effect,  these  rules 
will  represent  estimating  relationships  for  software  develop- 
ments with  an  efficient  allocation  of  resources.  Note  that 
we  are  Implying  that  much  of  the  variation  In  previous  esti- 
mating techniques  Is  the  result  of  mixing  In  the  same  data 
base  those  software  developments  which  have  Inefficient  with 
those  which  have  efficient  allocations  of  man-hours  among 
life-cycle  phases. 


We  also  developed  sone  new  aggregated  estimating  relationships. 

These  are  sumnarlzed  In  Sec.  3 of  this  volume  and  Include: 

1.  The  relationships  of  aggregate  software  man-hours  to  key 
driving  variables  such  as  size,  language,  capacity  constraints, 
and  requirement  changes  (ECPs)  during  development. 

2.  The  relationships  among  different  measures  of  the  size  of 
software  such  as  number  of  lines  of  source  code,  size  of  Part 
II  Specifications,  and  number  of  object-code  instructions. 
Conversion  ratios  between  object  and  source  code  for  different 
languages  and  machines  have  been  developed. 

3.  The  magnitude  of  maintenance  activities  and  the  separation  of 
error  correction  from  product  Improvement.  Examples  are  given 
In  which  product  improvement  has  been  the  major  consumer  of 
resources  during  maintenance.  Relationships  for  estimating 
error  rate  were  examined,  and  the  difference  In  maintenance 
manning  requirements  for  source  and  object  programs  was 
quantified. 

Detailed  derivations  and  measures  of  the  quality  of  the  relationships  are 
contained  In  Sec.  6 of  Vol.  I. 

1.4  RECOMMENDATIONS  FOR  FUTURE  WORK 

On  the  basis  of  this  study,  we  believe  that  the  following  efforts 
will  best  contribute  to  the  complicated  job  of  estimating  the  cost  of 
software. 

First,  and  of  utmost  Importance,  Implement  a cost  reporting  system. 

If  previous  systems  had  been  Implemented,  we  would  have  had  a usable  cost 
data  base  today.  We  hope  our  suggested  elements  are  the  basis  for  the 
approved  cost  reporting  system,  as  we  believe  they  avoid  some  of  the 
impediments  to  acceptance  by  contractors,  and  offer  a great  deal  of  con- 
trol for  the  Program  Offices.  However,  the  Important  thing  is  to  Imple- 
ment some  system.  Furthermore,  we  recommend  that  the  use  of  the  reporting 
system  be  a contractual  requirement . 
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Secondly,  we  recommend  that  the  Air  Force  continue  the  study  of  the 
PARMIS  data  base  begun  In  this  contract.  PARMIS  Is  the  only  readily 
available  data  base  In  which  the  relations  (trade-offs)  between  phases 
can  be  examined.  Although  our  results  to  date  are  quantitatively  incon- 
elusive,  because  of  the  small  sample  extracted  from  the  data  base,  the 
existence  of  the  relations  has  been  demonstrated.  Because  the  programs 
described  in  PARMIS  are  uniform  as  regards  Important  cost-driving  param- 
eters such  as  computer  language  and  type  of  applications,  and  because 
PARMIS  and  the  AFDSDC  organization  make  It  possible  to  obtain  very  de- 
tailed information  on  the  programs,  we  believe  that  further  study  of  the 
PARMIS  data  base  affords  a good  opportunity  to  quantify  the  relations 
among  phases. 

The  development  process  for  business-application  software  at  AFDSDC 
is  similar  to  that  for  major  defense  system  software.  Therefore,  the  func- 
tional forms  of  the  relationships  between  the  costs  of  life-cycle  phases, 
if  not  the  relationships  themselves  (with  some  scaling  rules),  can  be 
used  for  defense  system  applications.  It  is  most  Important  to  quantify 
these  relationships  because  they  are  the  key  to  identifying  proper  resource 
allocations  among  the  life-cycle  phases  and  thereby  achieving  risk  reduc- 
tion in  software  developments. 

Thirdly  and  concurrently,  a systematic  collection  of  data  from  cur- 
rent and  completed  projects  sponsored  by  Program  Offices  should  be  ini- 
tiated. Although  this  is  expensive  and  time-consuming,  we  cannot  afford 
to  wait  10  to  15  years  until  the  data  from  a reporting  system  (Implemented 
today  on  new  software  projects)  Is  usable  for  cost  estimation. 


2000  projects  are  included  In  the  data  base,  from  which  we  gathered  data 
on  20.  Most  of  the  data  base  remains  to  be  examined. 


17 


REPORTING  SYSTEM  ELEMENTS 

The  first  goal  of  Che  study  was  to  develop  a recommended  set  of  ele- 
ments for  a software  reporting  system  around  which  a uniform  data  collec- 
tion system  can  be  designed.  We  were  asked  to  identify  only  the  elements 
of  a system.  The  implementation  of  the  system — forms,  formats,  informa- 
tion flows,  etc. — was  beyond  the  scope  of  our  study. 

Four  primary  goals  guided  our  thinking  in  selecting  Che  reporting 
system  elements.  The  system  should: 

1.  Provide  for  cost  control  by  Program  Offices.  The  major  short- 
run  benefit  of  the  reporting  system  will  be  the  visibility 
given  to  the  Program  Offices  for  controlling  software  costs. 

2.  Provide  a data  base  for  better  cost  estimates.  The  long-run 
benefit  of  the  reporting  system  will  be  the  development  of  a 
software  cost  data  base  with  consistent  definitions.  The 
necessity  of  such  a data  base  for  developing  accurate  Cost 
Estimating  Relationships  Is  evident. 

3.  Have  a reasonable  chance  of  wide  acceptance  by  contractors. 
Perhaps  earlier  data  collection  systems^  were  not  Implemented 
because  they  asked  for  too  much  detail.  There  must  be  a 
trade-off  between  (1)  the  detailed  data  desirable  for  support- 
ing cost  estimation,  (2)  the  more  aggregated  information 
suitable  for  cost  control,  and  (3)  the  even  more  aggregated 
Information  that  contractors  will  readily  provide.  The  wider 
the  difference  between  (1)  and  (3),  the  more  likely  that  con- 
tractors will  resist  providing  the  Information. 

A.  Be  based  on  measurable  items.  Detailed  costs  should  be 
gathered  around  typical  contractor  work  packages  and  not 
artificial  groupings  of  costs.  Furthermore,  these  work  pack- 
ages should  be  standardized  so  that  comparisons  can  be  made 
between  programs. 
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Using  these  goals,  we  developed  the  following  three  principles  which 
have  guided  our  selection  of  elements: 

1.  Resource  consumption  should  be  related  to  progress  towards 
software  product  completion.  This  is  the  single  most  impor- 
tant item  for  cost  control. 

2.  The  level  of  detail  should  not  be  greater  than  specifically 
needed  for  cost  control  and  the  prediction  of  future  resource 
consumption. 

3.  Data  requested  should  relate  to  the  actual  development  pro- 
cess and  not  require  artificial  allocations. 

The  reporting  system  elements  were  defined  using  these  principles 
as  a guide.  Conceptually,  these  elements  have  three  dimensions:  the 
software  end  items  being  developed,  the  resources  being  consumed,  and 
the  phase  of  the  software  life-cycle  in  which  the  resource  consumption 
takes  place.  All  three  dimensions  are  Important  for  cost  control  and 
cost  estimation.  Also  of  great  importance  is  to  collect  information  on 
the  software  product's  degree  of  completion  along  with  the  resources  con- 
sumed. Only  then  will  the  relationship  between  resources  and  product  be 
established. 

This  summary  presentation  of  the  reporting  system  elements  will 
address  the  following  topics;  Life-Cycle  Phases,  Product  Status,  Soft- 
ware End  Items,  Reporting  System  Elements,  and  the  feasibility  of  quick 
Implementation  by  using  an  existing  reporting  system  (AFSCM  173-4).^  For 
a more  complete  development,  the  reader  is  referred  to  Sec.  8 of  Vol.  I. 

2.1  LIFE-CYCLE  PHASES 

The  following  "milestone"  definitions  are  adopted  for  the  life- 
cycle  phases: 
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Phase 


Beginning  Milestone 


Analysis 

Design 

Coding  and  Checkout 
Internal  Test  and  Integration 
Qualification  Test 
Installation 
Operations  and  Support 


Contract  Award 

Preliminary  Design  Review  (PDR) 

Critical  Design  Review  (CDR) 

Source-Deck  Baseline 

First  Preliminary  Qualification  Test  (PQT) 

Physical  Configuration  Audit  (PCA) 

Initial  Operational  Test  and  Evaluation 
(lOT&E) 


It  is  important  to  note  that  milestone  rather  than  activity  defini- 
tions are  being  used  for  the  phases.  Thus,  for  example,  if  some  coding 
is  done  before  CDR  it  is  counted  as  part  of  the  design  phase.  How  the 
resources  consumed  will  be  separated  by  activity  will  be  discussed  in 
Sec.  2.4. 


Secondly,  the  level  of  detail  available  changes  with  the  phase. 
Starting  with  PDR  and  ending  with  PCA,  developments  are  reported  by  Com- 
puter Program  Configuration  Items  (CPCIs) . By  CDR  the  CPCIs  have  been 
further  disaggregated  into  Computer  Program  Components  (CPCs)  and  informa- 
tion by  CPC  is  possible  until  PCA.  The  CPCs  are  themselves  made  up  of 
modules  of  code,  and  information  to  this  level  is  possible  in  the  coding 
phase. 


The  proposed  reporting  system  will  gather  information  by  phase  and 
the  potential  level  of  detail  will  differ  with  each  phase.  The  level 
of  detail  desired  and  the  relationship  of  the  resource  consumption  by 
phase  to  resource  consumption  by  activity  are  specified  subsequently. 

2.2  SOFTWARE  PRODUCT  STATUS 

There  is  no  sense  in  reporting  resource  consumption  to  a level  of 
detail  greater  than  that  describing  the  development  status  of  the  software 
product.  Hence,  we  first  defined  data  collection  requirements  for  infor- 
mation on  the  software  product,  independent  of  the  resources  consumed. 


20 


These  Include  the  following  for  each  CPC:  Descriptive  Information,  Deve- 
lopment Method,  Milestone  Information,  Code  Size  Measurements,  Code 
Changes,  and  Code  Structure.  Also  requested  is  Information  on  the  soft- 
ware development  as  a whole:  milestones,  number  of  pages  of  specifica- 
tions, number  of  test  procedures,  etc.  Details  are  contained  in  Tables 
8. 1-8. 3 of  Vol.  I. 

Reporting  requirements  are  related  to  phases,  with  single  reports 
required  at  the  milestones  separating  phases  and  recurring  reports  con- 
cerning code  development  status  (baselining  of  source  decks) , changes  to 
code  during  the  qualification  test  phase,  and  code  status  during 
maintenance. 

Milestone  reports  become  more  detailed  as  the  development  progresses, 
culminating  in  a very  detailed  description  of  the  product  at  PCA.  Recur- 
ring reports  are  more  aggregated  and  do  not  require  detailed  documenta- 
tion. For  example,  code  status  can  simply  be  reported  by  delivering  a 
copy  of  the  source  deck  for  each  module,  upon  completion  of  coding  and 
unit  testing. 

This  continued  reporting  of  source  code  completion  will  provide  an 
almost  continuous  measure  of  product  completion  (e.g.,  30%  of  the  source 
code  has  now  been  written  and  debugged).  Furthermore,  actuals  can  be 
checked  against  estimates  to  see  if  man-hour  consumption  is  in  line  with 
estimates,  product  size  is  within  projections,  and  coding  is  on  schedule. 

We  feel  this  innovation  is  the  most  important  product  status  data 
item  recommendation.  To  date  no  reliable  data  on  product  completion  is 
available  to  the  Program  Office  between  CDR  and  PQT.  This  void  of  infor- 
mation has  prevented  the  early  recognition  of  problems  and  hence  the 
implementation  of  early  corrective  measures.  The  requirement  for  source- 
deck  delivery  against  a prearranged  schedule  will  fill  this  void. 
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The  milestone  reports,  while  operationally  not  as  meaningful,  will 
provide  a detailed  and  standardized  statement  of  what  was  produced.  The 
collection  of  reports  will  provide  a history  of  the  development.  The 
entire  set  of  data  will  provide  the  product  description  data  base  neces- 
sary for  developing  Cost  Estimating  Relationships. 

2.3  SOFTWARE  END  ITEMS 

A standard  list  of  software  end  Items  has  been  selected  from  which 
we  propose  that  Computer  Program  Components  be  selected  (Table  2.1).  We 
recognize  that  such  a list  Is  tentative  and  will  undergo  change  with  use. 
However,  the  need  for  a standardized  list  to  disaggregate  major  defense 
system  software  Into  recognizable  pieces  Is  critical.  Although  each  soft- 
ware development  is  unique,  the  parts  are  often  common  to  many  develop- 
ments. Collection  of  data  to  a standardized  list  will  eventually  enable 
software  cost  to  be  estimated  at  the  CPC  level,  much  as  hardware  Is  now 
estimated  on  the  component  level. 

2.4  DEFINITIONS  OF  REPORTING  SYSTEM  ELEMENTS 

Software  resources  were  divided  Into  the  following  categories  dur- 
ing the  study.  (We  focused  on  contractor  costs.) 

Contractor 

Direct  Technical  and  Support  Labor 

Computer  Resources 

Unique  Facilities  and  Equipment 

Overhead 

Support  Labor 

Materials 

Travel 

G&A  and  Fee 

Government 

Air  Force  Labor 

Military 

Civilian 

Technical  Assistance  Contractor  Labor 
Government  Furnished  Equipment  and  Facilities 
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TABLE  2.1 


SUGGESTED  LIST  OF  COMPUTER  PROGRAM  COMPONENTS 
(Table  8.4  of  Volume  1) 


1 SUPPORT  SOFTWARE 

1.1  EXECITIVE 

1.1.1  Computer  Resource  Manager 

1.1.2  Computer/Peripheral  Interface 

1.1.3  Computer /Operator  Interface 

1.1.4  Computer /Computer  Interface 

1.1.5  Computer/Special  Device  Interface 

1.1.6  System  External  Interface 

1.1.7  System  Failover  and  Recovery 

1.1.8  Performance  Monitoring 

1.2  UTILITIES 

1.3  LANGUAGE  PROCESSORS 

1.3.1  Assembler 

1.3.2  Compiler 

1.3.3  Interpreter 

1.3.4  Translator 

1.4  LOADERS 

1.4.1  Bootstrap  Loader 

1.4.2  Linkage  Editor/Loader 

1.5  HARDWARE  DIAGNOSTICS 

1.6  SYSTEM  SIMULATIONS,  TESTS,  AND  EXERCISES 

1.6.1  Operational  Scenario 

1.6.2  Control  and  Sequencing 

1.6.3  Data  Collection 

1.6.4  Data  Reduction  and  Analysis 

1.7  SOFTWARE  DEVELOPMENT  AIDS 

1.8  PROJECT  MANAGEMENT  AIDS 

1.8.1  Schedule  Maintenance 

1.8.2  Financial  Accounting 
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TABLE  2.1  (Continued) 


2 APPLICATIONS  SOFTWARE 

2.1  AVIONICS  SOFTWARE 

2.1.1  Mission  Planning 

• Preplanned  Mission  Evaluation 

# Real-Time  Mission  Modification 

• Steering  Coordination 

e Weapon  Delivery  Coordination 

e Waypoint  Sequencing  and  Mode  Selection 

2.1.2  Navigation 

# TACAN 

# LORAN 

e Doppler  Radar 

e Inertial  Reference  Unit 

e Auxiliary  Attitude  Reference 

a Air  Data  Computations 

a Kalman  Filter 

2.1.3  Aircraft  Steering 

a Course  Select 

a Manual  Course 

a Instrument  Landing  System 

a Airborne  Instrument  Landing  & Approach 
a Data  Link 

a TACAN 

2.1.4  Flight  Controls 

a Roll,  Pitch,  and  Yaw  Controls 

a Velocity/Acceleration  Control 

a Air  Induction  Control 

a Energy  Management  and  Control 

a Cockpit  Environment  Control 

a Crew  Flight  Input 

2.1.5  Weapon  Delivery 

a Continuous  Computation  of  Impact  Point 

a Weapon  Miss  Distance  Computation 

a Automatic  Release  Control 

a Visual  Release  Control 

• Stores  Management  and  Control 
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TABLE  2.1  (Continued) 


2.1.6  Sighting.  Dealgnatlon.  end  Plxtakina 

e Forward  Looking  Radar 

e Astro  Tracker 

e Laser  Spot  Seeker 

e Low  Light  Level  Television 

e Forward  Looking  Infrared  Radar 

e Low  Altitude  Radar  Altimeter 

e Tracking  Handle 

e Electro-Optical  Sighting 

e Visual  Sighting 

e Fixtaking 

2.1.7  Display  Control 

e Heads-Up  Display 

e Navigation  Display 

e Sensor  Display 

e Data  Display 

2.1.8  Data  Entry/Retrieval 

e Mission  Entry 

e Aircrew  Panel  Control 

e Data  Base  Access 

2.1.9  Coasninlcat Ions 

2.1.10  Electronic  Countermeasures 

e Threat  Naming 

e Electronic  Warfare 

e Penetration  Aids 

2.2  CCHIMUNICATIONS,  COMMAND,  CONTROL,  AND  INTELLIGENCE  SOFTWARE 

2.2.1  Data  Acquisition 

e Sensor  Control 

e Signal  Processing 

e Data  Transfer 
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TABLE  2.1  (Continued) 


2.2.2  Data  Processing 

• Mission  Control 

a Data  Identification 

s Data  Reduction 

s Data  Manipulation 

2.2.3  Operator  Analysis  and  Decision  Maklnp; 

s Mission  Management 

s Data  Display  and  Monitoring 

s Operator  Data-Entry  and  Control 

2.2.4  Response  Generation 

s Communications  Switching 

s Message  Processing 

e Report  Generation 

2.2.5  Data  Storage  and  Retrieval 

3 DATA  BASE 

3.1  FILES 

3.1.1  On-Line  Updatable  Files 

3.1.2  Internal  Files 

3.1.3  System-Generated  Files 

3.1.4  Remote  Data  Base  Files 

3.2  INDEXES 

3.3  TABLES 

3.4  PROGRAM  SUPPORT  LIBRARY 

3.4.1  Program  Library 

3.4.2  Macro  Library 

3.5  MAINTERANCE  ROUTINES 
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Our  recommended  set  of  resource  elements  is  listed  In  Table  2.2. 

Note  that  they  are  oriented  to  the  activities  required  to  develop  the 
software  rather  than  the  milestone-defined  life-cycle  phases.  The  rela- 
tionship of  these  resource  elements  to  life-cycle  phases  Is  given  In 
Table  2.3.  It  was  extremely  Important  to  distinguish  between  the  activity 
and  milestone  uses  of  these  terms,  vnille  It  Is  the  allocation  of  resources 
to  activities  that  Is  Important  to  the  efficient  development  of  software. 

It  Is  the  milestone  definitions  that  have  often  been  used  when  disaggregated 
costs  have  been  reported. 

Note  that  the  reporting  levels  as  well  as  the  elements  themselves 
have  been  selected  to  avoid  artificial  allocations  of  resources.  For 
example,  the  allocation  of  functions  to  CPCIs  Is  Included  In  Analysis 
and  reported  at  the  contract  or  system  level  rather  than  attempting  an 
allocation  to  CPCIs.  Also  there  Is  no  attempt  to  separate  redesign  from 
recoding  during  Integration  and  Testing.  These  error-response  activities 
are  merely  incorporated  Intd  the  Integration  activity. 

Also  note  that  there  Is  no  provision  for  product  Improvement  during 
Operations  and  Support.  It  Is  our  recommendation  that  product  Improve- 
ment be  reported  as  a separate  development  so  that  descriptions  of  the 
improvements  can  be  determined  and  their  costs  Identified.  Only  then 
can  a cost/benefit  determination  be  made. 

The  elements,  phases,  and  reporting  levels  are  then  related  to  the 
contractor  resources  as  shown  In  Table  2.4.  The  different  types  of  re- 
sources (man-hours,  computer  hours,  and  costs)  ace  Identified  and  their 
reporting  frequency  and  level  specified. 
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TABLE  2.2 

REPORTING  SYSTEM  ELEMENTS 
(Table  8.6,  Volume  I) 


Activity 
CPC  Uvl; 

Design 
Coding 
Integration 
Installation 
Maintenance 
CPCI  Uvei; 


Definition 

All  nan-hours  for  particular  activities  that  can  be  Identified  to  the  CPC  level 
Detailed  CPC  design 
Coding  and  initial  checkout  of  CPCs 

Kedesign  and  recoding  In  reaponae  to  test  error  detection 

Modification  of  code  for  site-specific  application 

Coding  and  Integration  in  response  to  error  reports  (OKPs) 

All  resources  for  particular  activities  related  to  the  CPCI  and  not  aooicnable 
to  CPs 


Design 

Testing 

Independent  VAV 
(k>ntract  or  Systee  Level; 


CPCI  design,  Interfaces,  and  allocation  of  functions  to  CPCa  and  nodelea 

Defining  and  carrying  out  CPCI-level  software  tests 

Independent  verification  and  validation  of  CPCIs 

All  resources  for  particular  actlvltiea  related  to  the  contract  or  ayot«  Mid 
not  assignable  to  CPCs  or  CPCIs 


Analysis 


* 

Testing 


Studies  to  resolve  conceptual  problens  and  denonstrate  capability  to  *001 
requireiscnta.  Including  algoritha  developaent,  allocation  of  functions  to 
CPCIs,  etc. 

Testing  of  software  functions  at  systen  level,  including  hardware  Interface 


Independent  VAV  Independent  verl|ication  and  validation  at  syatea  level 

Hanagencnt 


Engineering  All  nanagenent  personnel  assigned  to  oversee  the  technical  quality  of  the  soft- 

ware product,  including  directing  software  developaent,  directing  the  prepara- 
tion and  review  of  software  portions  of  technical  doctMonta,  preparation  for  and 
attendance  at  technical  review  nectlngs,  on-slte  support  during  operations,  fault 
Isolation,  etc. 

Configuration  All  nanagenent  personnel  assigned  to  oversee  the  naintenance  of  baselined  soft- 

ware and  software  specifications  and  to  docunent  and  Incorporate  approved 
changes  to  the  baseline 

* 

Project  Other  nanagenent  activities  directly  associated  with  the  software  product,  in- 

cluding contract  nanagenent,  coot  and  schedule  nanagenent,  buslneaa  and  adninla- 
tratlon  planning,  directing  and  controlling  the  project,  other  than  these  tech- 
nics! and  configuration  control  tasks  Identified  above. 


Training 


Preparation  of  course  naterial,  denonstration  and  conduct  of  training  courses  If 
contracted,  etc. 


Docosntatlon:  Writing,  editing,  publishing*  reproduction,  and  disseninatlon  of  software  docu- 

■ents  required  by  the  CMU,.  Could  be  reported  at  CPCI  level  for  nore  detail. 

Technical  Part  1 and  Part  11  Specifications,  Test  Plans,  Uker*a  Manuals,  specified  techni- 

cal reports,  etc. 

Configuration  Version  descriptions,  configuration  index,  change  statiM  reports,  specification 

change  notices,  etc. 


Prograa  ^ Software  developeent  plan,  coat  perforwsnce  reports,  prograe  schedules,  prograe 

Managseent  ellestonea,  etc. 


Support 


Mon-technical  personnel  totally  assigned  to  the  software  portion  of  the  contract 
such  as  key  punchers,  secretaries,  etc. 


* 


These  iteeo 


are  sowetlnes  difficult  to  separate  fron  hardware. 
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TABLE  2.3 

REPORTING  SYSTEM  ELEMENTS  AND  LIFE-CYCLE  PHASES 


(Table  8.14  of  Volume  I) 


Internal  Taat  aad  Qualification 

Raportlnf  Syotam  Aealyaia  Oaaltn  Co4inf  4 Checkout  Integration  Tent  Om'**!****  * 

Clementa  Contract  Award  t»POIt  PDR  tfCDI  CM  to  Source  gaaellne  Source  taaelloe  to  PQT  PQT  to  PCA  Inatallatten  tuMort 


1 

! CPC  Leva  1 

Ovtlgn 



Authorized  Early 

Dealgn 

Detailed 

Dealgn, 

Flowchart 

Authorized  lete 

Dealgn 

Ceding 

Authorized  Early 
Coding 

Authorized 

Urly 

Coding 

Coding  and  Debugging 

1 Iniogratlon 

Error  teaponac 

Error 

Raeponee 

1 Inatallatlon 

Site-Specific 

Chaagea 

Ralntananc* 



Priority  Error 
Corractioa 

' rpci 

1 

! 

i 

Authorized  Early 

Dealgn 

Module.' 
Funct Ion 
Allocation 
Interface 
Dealgn 

1 

1 

Teat ing 

Teat 

Procedurea 

CPCl  Teata 

CFCI 

Teete 

1 

Independent 

V4V 

V4V  Teata 

Svatea  Level 

1 

Anelyala 

t 

1 

Special  Studtaa 

Authorized 

Late 

AnalyKra 

Special  I 

Studiea  1 

1 

[ 

1 Teat ing 

Teat  Plan 

Syaten 

Teat 

Procedurea 

Syaten 

Teata 

Contractor 

ofc-site 

Teata  of 
Changaa 

1 

Contractor 
Qualification  1 
Teat  of  Error 
Correction 

1 Independent 

1 V4V 

1 

1 

Requlremente. 

Val  (d.H  l>m 

V4V  PUn 

V4V  Procedurea 

V4V 

Teata 

On-Site 
Qualification 
Teat  of 

Changea 

Independent 
Qualification  | 

Taatlng  of 

Error  CorrcctlonI 

1 

Management : 

1 Eng. 

I 

1 

I 

X 

X 

X 

X 

! Conflg. 

X 

« 

X 

X 

I 

X 

1 Project 

X 

X 

X 

« 

X 

j Training 

! 

Training  < 

Flan 

. ! 

Training 

Manual 

FrapdTAtlen 

Initial 

Oner 

TralnlM 

Training  and 
DMonatratlon 

Dorumentat  le^ 

1 

Technical! 

» 

X 

X 

« 

X 

X 

Conflg. 

X 

X 

« 

X 

» 

X 

Pro.Ngat 

' 

X 

X 

I 

X 

I 

X 

Support  1 

. 

X 

X 

X 

X 

X 

X 

I 


1 
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TABLE  2. A 


CONTRACTOR  REPORTING  OF  RESOURCE  CONSUMPTION 
(Table  8.18  of  Volume  I) 

In  each  category,  estimated  costs  are  to  be  reported  once,  at  the  begin- 
ning of  the  contract,  except  as  noted.  Actual  costs  are  to  be  reported 
monthly,  except  as  noted. 

Notes 

Direct  Labor  (in  dollars  and  in  man-hours)  1,  2,  3 

Computer  Resources 

Computer  Hours  1 

Investment  A 

Operations 

Unique  Facilities  and  Equipment 

Investment  4,  5 

Operations  6 

Overhead  Rate 
G&A  and  Fee  Rates 

j 

NOTES: 

1.  Disaggregated  by  computer  program  component  as  in  Table  2.3 

2.  Estimates  disaggregated  by  skill  category  as  In  Sec.  8. 6. A of  Vol.  I. 

3.  Estimates  revised  at  PDR  and  at  CDR. 

4.  Actuals  reported  when  Incurred. 

5.  Disaggregated  by  type  of  facility. 

6.  Disaggregated  by  type  of  facility  or  equipment. 
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Section  8 of  Vol.  I also  specifies  reporting  requirements  for  unique 
facilities  and  equipment,  computer  system  description  (Table  8.16  of  Vol. 

I)  and  utilization  (Table  8.19  of  Vol.  I),  and  gives  a standardized  list 
of  skill  categories  including  job  descriptions  and  current  salary  infor- 
mation (Appendix  C of  Vol.  I). 

2.5  FEASIBILITY  OF  IMPLEMENTATION 

Specifying  reporting  system  elements  is  only  part  of  the  require- 
ment for  a better  software  resource  consumption  data  base.  Implementa- 
tion is  the  other  (and  perhaps  most  important)  part.  We  therefore 
attempted,  successfully,  to  match  the  reporting  system  elements  to  an 
extended  version  of  the  computerized  reporting  system  defined  in  AFSCM 
173-4.^ 

Without  the  extension,  defined  in  Ref.  4,  the  Program  Breakdown 

Structure  of  Ref.  3 is  too  aggregated  for  reporting  software  resource 

consumption.  Briefly,  the  extension  expands  the  software  reporting  code 

* 

42xx  to  accommodate  a more  detailed  breakout.  First,  the  code  is  expanded 
to  include  interim  standard  Program  Breakdown  Codes  (PBCs) , which  have 
the  form 

s421xxy 

where  s is  a letter  that  identifies  the  software's  supplier 

421  identifies  the  element  as  a software  product 

XX  is  an  alphanumeric  code  that  designates  the 
software  type 

y when  used,  is  a letter  (A,  B,  ...,  F)  that  identifies 
the  computer  program  life-cycle  phase  to  which  the  ele- 
ment applies;  if  y is  not  used,  the  element  is  presumed 
to  encompass  all  such  phases  covered  by  the  contract 

— _ 

This  description  reproduces  the  main  parts  of  the  description  in  Ref.  4. 


31 


In  addition,  the  following  software  analogs  for  the  standard  WBS 
Level  2 categories  are  Included: 

PBC  Software  Support  Element  Name 

4210  Computer  programs  (coding,  checkout,  purchase,  or  rental 
costs  only) 

4220  Software-peculiar  training 

4240  Equipment  required  specifically  for  software  development 

4250  Testing  of  software 

4260  Software-peculiar  management  and  engineering 

4270  Software  documentation 

4285  Software  development  and  maintenance  facilities 

4200  Summary  of  all  software-related  costs 

Furthermore,  the  extended  code  allows  for  the  further  designation 
of  the  product  by  providing  three  or  more  positions  (after  /)  to  desig- 
nate segment,  function,  CPCI,  etc.  Thus,  s421xxy/ABC  designates  resource 
consumption  during  phase  y for  CPC-xx,  CPCl-C,  of  functional  area 
B and  Segment  A. 

We  modified  the  y designator  to  refer  to  specific  reporting  ele- 
ments rather  than  life-cycle  phases.  Life-cycle  phase  Information  can  be 
recorded  by  summarizing  the  data  at  the  milestones  which  separate  the 
individual  phases. 

With  this  modification  and  the  above  extended  system  definitions, 
we  assigned  codes  to  each  of  the  resource  elements  shown  in  Table  2.2. 
Results  are  reported  In  Tables  8.21  and  8.22  of  Vol.  I. 
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The  ability  to  use  this  extended  reporting  system  demonstrates  that 
our  recommendations  can  be  Implemented  without  the  research  costs  and 
delays  associated  with  data  base  software  development. 

In  conclusion,  we  feel  that  Implementation  of  this  (or  a similar) 
reporting  system  Is  of  prime  Importance.  We  feel  that  we  have  Integrated 
the  three  dimensions  of  the  reporting  system  (life-cycle  phases,  end  items, 
and  resource  consumption)  Into  a workable  and  consistent  set  of  reporting 
elements.  Furthermore,  we  have  Incorporated  reporting  requirements  for 
product  description  and  status  (Sec.  2-2)  that  will  go  a long  way  towards  elding 
the  Program  Offices  In  cost  and  product  development  control.  We  have 
tried  to  reach  the  right  balance  between  the  Government's  needs  for  data 
and  acceptance  by  contractors. 

We  are  confident  that  if  this  reporting  system  is  Implemented,  the 
Program  Offices  will  have  better  project  control  and  the  software  com- 
munity will  eventually  have  a good  resource  requirements  data  base. 
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3 RESOURCE  ESTIMATING  RELATIONSHIPS 

As  mentioned  In  Sec.  1.3,  we  were  able  to  develop  some  new  resource 
estimating  relationships,  despite  the  difficulties  discussed  in  Sec.  1.2. 
The  results  address  parts  of  the  software  resource  estimating  problem 
and  do  not  form  a complete  model.  Rather  they  are  the  development  of 
some  of  the  underlying  relationships  that  precede  the  development  of 
accurate  cost  models. 

This  section  summarizes  the  results  in  four  areas:  (1)  similarities 
between  "ADP"  and  defense-system  software,  (2)  aggregated  relationships, 
(3)  alternative  product  measures,  and  (4)  maintenance  activities.  A 
detailed  discussion  of  the  development,  applications,  and  equation  deriva- 
tions is  contained  in  Sec.  6 of  Vol.  I.  Our  principal  sources  of  data 
were  the  ADP  Resource  Estimating  Procedures  (ADPREP)  study, ^ the  Air  Force 
Satellite  Control  Facility  (SCF)  Program  Office,  some  GRC  Internal  data, 
and  some  data  from  the  open  literature  dealing  with  software  reliability. 
The  number  of  programs  retrieved  from  PARMIS  was  too  small  to  make  it  a 
statistically  significant  data  source. 

3.1  SIMILARITIES  BETWEEN  "ADP"  AND  DEFENSE-SYSTEM  SOFTWARE 

One  of  the  first  considerations  is  the  reasonableness  of  using  data 
on  business-applications  or  "ADP"  software  to  study  and  predict  the  costs 

it 

of  defense-system  software.  In  some  phases,  the  cost  of  software  deve- 
lopment obviously  depends  upon  the  application. 

Figure  3.1  Illustrates  how  various  types  of  software  development 
relate  to  one  another  in  terms  of  development  effort  for  specified  size. 
Generally,  the  defense  system  programs  require  more  man-hours.  However, 
commercial  and  defense-system  programs  exhibit  similar  trends,  with  some- 
what more  dispersion  in  the  commercial  data.  We  believe  that  the  pro- 
gramming processes  used  to  develop  commercial  and  defense-system  programs 


This  is  especially  important  with  the  reliance  on  PARMIS  data  for  future 
studies  recommended  in  this  report. 
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DEVELOPMENT  EFFORT,  man-months 


• DEFENSE  SYSTEM  DATA*-*- 
□ AOPREP  DATA  (MIXED) 

A PARMIS  DATA  (BUSINESS  SYSTEMS) 


ts. 


5 


Figure  3.1.  Size  and  Effort  in  Three  Data  Bases  (Figure  6.1  of  Vol.  I) 


are  similar,  and  that  the  Increased  unit  effort  for  defense-system  pro- 
grams can  be  explained  by  their  complexity;  this  Is  reflected  in  the  coef- 
ficients of  cost  estimating  relationships.  We  believe  that  the  functional 
relationships  between  the  components  of  the  development  activity  and  the 
variables  that  explain  effort  are  similar  for  either  type  of  system. 

Thus,  valuable  insights  can  be  obtained  from  the  study  of  "commercial" 
or  "ADP"  developments.  They  also  offer  the  advantage  of  clearly  distin- 
guishing the  costs  of  software  from  computer  equipment  (or  other  "system 
level")  costs. 

Additional  observations  as  to  why  differences  shown  In  Fig.  3.1  are 
exaggerated  can  be  found  In  Sec.  6.2  of  Vol.  I. 

3.2  AGGREGATED  RELATIONSHIPS 

Figure  3.1  shows  an  apparent  relationship  between  the  size  of  the 
software  and  the  effort  required  to  develop  It.  However,  the  data  varies 
so  much  that  a relationship  based  solely  on  such  a trend  Is  not  a useful 
cost  estimate.  Resource  consumption  trade-offs  between  the  life-cycle 
phases  are  one  explanation  of  the  variation.  Other  variables  can  also 
be  the  cause,  and  we  explore  several  In  Sec.  6.3  of  Vol.  I. 

Programming  Language.  A number  of  authors  have  discussed  the  rela- 
tionship between  language  and  programmer  productivity.  We  also  observed 
this  relation  In  our  study,  developing  the  following  relationship: 

T.  * 0.04N°'^^0.54^ 
dev 

where  ^dev  * development  time,  man-months 

N - nundjer  of  object-code  Instructions 
L - 0 If  assembly  language 

» 1 if  high-level  language  i 

As  can  be  seen,  the  use  of  higher-order  language  Is  more  productive,  l.e., 
uses  less  man-months,  than  the  use  of  assembly  language.  The  quality  of 
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this  relationship  Is  not  high  (low  R ),  but  It  was  sufficient  to  estab- 
lish the  Importance  of  programming  language. 

Hardware  Constraint.  In  an  earlier  GRC  effort,^  we  tested  the 

effect  of  core-memory  limitation  on  programmer  productivity.  The  data 

A A 

had  been  allocated  to  three  subsections  (design,  coding,  and  testing) 
and  equations  were  developed  for  each. 

The  results  obtained  are  shown  In  Table  3.1.  In  each  case,  effort 
(man-years)  Is  approximately  linear  with  number  of  object  Instructions. 


TABLE  3.1 

HARDWARE-CONSTRAINT  EQUATIONS 
(Table  6.1  of  Volume  I) 

T.  * 0.0030N^*°^5.64^ 
des 

T . - 0.000A9N^*^^8.33^ 

code 

T^  ^ - 0.0080N®'^^^5.15^ 
test 


where 


T T T 

des’  code’  test 


times  for  design,  coding,  and  test 
(man-months) 


iO  If  no  hardware  constraints 
1 If  more  chan  95%  of  the  core  Is  utilized 


R Is  a statistical  term  commonly  used  to  show  goodness  of  model  fit. 

The  closer  to  1,  the  better  the  fit.  R^  of  0.8  or  above  are  good  with 
0.6  to  0.8  marginal. 

These  breakouts  are  not  equivalent  to  the  phase  definitions  used  In 
the  current  study. 
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Hardware  constraints,  when  present.  Increase  cost  by  a factor  of  five  or 


more. 


Design  Changes.  Experience  with  software  developments  has  indi- 
cated that  changes  in  requirements  (through  ECPs)  can  be  an  explanation 
of  variations  in  product  costs.  Because  of  such  changes,  the  delivered 
product  size  understates  the  size  of  the  software  which  may  actually  have 
been  written.  Thus  development  effort  per  delivered  instruction  is  con- 
siderably overstated. 

Detailed  data  was  not  collected  in  sufficient  quantity  from  PARMIS 
to  analyze  the  direct  Impact  of  requirement  changes  on  individual  life- 
cycle  phases.  The  ADPREP  data^  did  report,  by  project,  the  number  of 
changes  made  during  development.  Unfortunately,  there  was  little  discus- 
sion of  the  nature  of  the  changes  and  no  attempt  to  quantify  the  impact 
of  an  individual  change.  Thus,  the  only  measure  of  change  activity  dur- 
ing development  was  the  cumulative  number  of  changes  Incorporated  into 
the  software. 

In  an  attempt  to  use  the  available  data,  it  was  assumed  that  each 
change  during  development  modified  some  average  fraction  of  the  software 
that  was  intended  for  delivery.  The  number  of  object-code  Instructions 
affected  by  a change  was  estimated  as  the  product  of  the  size  of  the 
delivered  code,  the  cumulative  number  of  changes  during  development,  and 
the  estimated  fraction  of  the  code  that  changed  with  each  requirement 
change.  It  was  hypothesized  that  effort  was  linearly  related  to  delivered 
program  size  and  to  the  amount  of  code  that  changed  during  development 
of  the  program.  This  was  represented  by  an  estimating  equation  that  in- 
cluded the  product  of  the  number  of  requirements  changes  and  the  size  of 
the  delivered  product,  as  well  as  the  size  of  the  delivered  product: 

T.  - 6.61  + 0.000678N  + O.OOOllH  * C 
dev 


where 


C ■ Number  of  changes  (e.g.,  ECPs) 


2 

The  R for  this  relationship  was  0.65,  which  is  encouraging.  The  data 
does  substantiate  the  cost  of  changes  during  development. 

An  application  of  a changing- requirements  cost  factor  is  to  develop 
an  initial  cost  estimating  technique  that  requires  the  user  to  estimate 
the  expected  number  of  changes  that  will  occur  during  the  project's  life- 
time. In  this  way  the  Impact  of  change  can  be  anticipated  and  explicitly 
incorporated  into  the  initial  estimate  of  costs. 

3.3.  ALTERNATIVE  PRODUCT  MEASURES 

Throughout  the  study,  an  attempt  was  made  to  relate  measures  of  the 
product  of  a software  development  to  the  effort  and  costs  associated  with 
that  product.  The  most  obvious  and  commonly  suggested  measure  is  software 
size,  expressed  in  terms  of  number  of  either  source- language  statements  or 
object  Instructions.  Each  seems  to  have  an  appropriate  application,  and 
it  is  Important  that  a consistent  measure  be  used  when  comparing  data  from 
various  projects.  Inconsistency  in  this  measure  of  program  characteristics 
is  probably,  by  Itself,  significant  in  explaining  variation  in  the  data 
reported,  since  it  could  vary  by  a factor  of  two  to  five. 

Even  if  measures  of  computer  program  size  can  be  converted  to  a 
normalized  form  (such  as  number  of  object  instructions),  another  possible 
source  of  inconsistency  exists.  It  has  been  argued  that  source-to-object- 
code  relationships  for  high-level  languages  are  invalid  because  differences 
in  programming  style  (i.e.,  coding  procedures  and  techniques)  will  result 
in  significant  variations  from  programmer  to  programmer. 

The  AOES  data  offered  an  opportunity  to  test  this  conjecture.  The 
data  base  contains  both  the  number  of  source-code  card  images  for  a large 
number  of  programs  and  the  number  of  object-code  instructions  generated  by 
a JOVIAL  compiler.  The  entire  set  of  JOVIAL  programs  which  were  considered 
ran  on  the  same  computer,  so  variations  Introduced  by  computer  instruction 
sets  are  removed.  Furthermore,  the  programs  perform  a wide  range  of 
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functions,  from  orbit  determination  to  compilation  and  operating  system 

functions,  and  are  written  by  many  authors.  These  AOES  programs  were  then 

used  to  test  the  claim  that  significant  variations  in  the  relationship 

between  the  instruction  counts  of  source-code  and  object-code  are  introduced 

by  differences  in  programmer  style.  Regression  analysis  was  used  with  the 

2 

rationale  that  a weak  linear  relationship  (i.e.,  low  R ) supports  the 
hypothesis.  The  following  equation  was  developed: 

N = -30.495  + 2.4N„ 
where,  as  before, 

N = number  of  object-code  instructions 
N„  = number  of  source-code  card  Images 

d 

2 

A very  strong  linear  relationship  (R  = 0.94)  was  found  between  source 
and  object  code  for  this  set  of  JOVIAL  programs.  These  results  indicate 
that  the  assertion  is  invalid;  i.e.,  programmer  stylistic  variations  in 
the  use  of  JOVIAL  are  minor.  Note  that  stylistic  variations  may  still 
be  an  Important  determinant  of  number  of  source  instructions. 

This  suggests  that  a table  of  conversion  factors  for  language  can 
be  compiled  for  various  programming  languages  and  host  computers.  These 
factors  could  be  applied  to  construct  normalized  measures  of  product  size. 
Table  6.2  of  Vol.  I contains  a compilation  of  such  data  based  on  the  in- 
formation used  in  this  study. 

Software  development  yields  other  products  besides  software,  notably 
documentation.  It  is  possible  that  a measure  of  documentation  can  serve 
as  a surrogate  for  product  size  measured  in  source- language  or  object- 
language  Instructions.  To  test  this  conjecture,  it  was  assumed  Chat  the 
size  of  the  documentation  describing  a project  would  be  related  to  the 
size  of  the  final  product,  so  long  as  common  documentation  requirements 


and  formats  applied.  Relaxing  this  last  constraint  would  make  any  com- 
parison meaningless. 

The  AOES  data  provides  Information  that  could  be  used  to  test  this 
conjecture.  For  each  of  the  many  programs  examined,  the  page  counts  for 
Part  II  Specifications  were  available,  as  well  as  measures  of  the  product 
size  In  terms  of  the  number  of  source  statements.  The  following  equation 
was  developed: 

Ng  = 1600Np‘^ 

where  Ng  « number  of  source-code  Instructions 

Np  > number  of  pages  of  Part  II  Specifications 

2 

The  relationship  has  a moderately  low  R value  (0.56)  that  indi- 
cates It  should  not  be  used  to  replace  existing  measures  of  software  size. 
However,  It  does  show  promise  and  this  type  of  relationship  should  be  fur- 
ther studied.  Such  a relationship  between  documentation  and  program  size 
could  be  used  to  check  estimates  of  program  size  at  the  time  of  CDR  when 
the  first  product-oriented  documentation  exists  In  draft  form.  If  the 
relationship  to  documentation  did  not  hold,  the  sizing  of  the  program 
might  then  be  subjected  to  further  review,  and  a possible  revision  of  pro- 
ject costs  and  schedule  might  be  Indicated. 

This  type  of  relationship  also  offers  promise  In  directly  calculat- 
ing the  cost  of  documentation.  Since  source  cards  are  related  to  number 
of  pages,  an  estimating  equation  for  documentation  could  be  calculated. 

3.4  MAINTENANCE  ACTIVITIES 

Few  previous  cost  estimating  studies  of  software  have  examined  this 
activity  of  the  life  cycle  to  any  significant  extent.  In  this  study,  it 
became  apparent  that  the  entire  process  was  very  poorly  understood.  The 
maintenance  activity  for  software  has  two  components:  (1)  error  correc- 
tion, and  (2)  lflq>rovement  or  refinement  of  the  software  product. 
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The  AOES  data  provides  some  insight  into  the  relative  proportions 
of  effort  devoted  to  error  correction  and  to  product  improvement.  The 
data  Includes  the  allocation  of  direct  technical  man-hours  during  mainte- 
nance for  the  AOES  project,  in  terms  of  the  activities  of  the  two  asso- 
ciate contractors:  one  spent  25  percent  of  its  man-hours  on  error  cor- 
rection and  75  percent  on  product  Improvement;  for  the  other  associate 
contractor,  the  division  was  33  percent  and  67  percent.  Thus,  for  both 
contractors,  most  of  the  effort  went  to  improvement  rather  than  error 
correction. 

The  point  is  that  a regular  pattern  of  modification  takes  place  and 
is  to  be  expected  during  the  operations  period.  However,  these  modifica- 
tions are  related  to  product  Improvement,  rather  than  the  initial  develop- 
ment's success  of  failure  in  meeting  original  requirements.  No  trade-off 
exists  between  development  and  improvement,  and  a decision  to  fund  the  improve' 
ment  should  be  based  solely  on  costs  and  benefits  of  the  improvement  itself. 

Resources  consumed  in  software  error  correction,  on  the  other  hand, 
are  clearly  driven  by  the  problems  reported  during  operation  of  a system 
and  the  response  times  required  to  correct  the  reported  errors.  Further, 
these  are  and  should  be  related  to  the  software  development  effort. 

The  technical  literature  on  software  reliability  indicates  that 
the  error  process  of  operational  software  exhibits  two  major  trends. 

First,  the  total  number  of  problems  encountered  with  a given  model  of 
software  is  limited,  and  the  limit  is  related  to  the  size  of  the  software. 
Second,  the  mean  time  between  error  detections  increases  with  use  and 
corresponding  error  correction. 

The  data  collected  for  the  AOES  project  were  at  a sufficient  level 
of  detail  to  allow  testing  of  some  of  these  assertions.  It  was  hypothe- 
sized that  the  number  of  problems  reported,  once  a release  of  the  com- 
puter program  became  operational,  would  be  related  to  the  size  of  the 
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computer  program  (residual  errors),  the  new  requirements  Introduced  with 
the  release  of  the  modified  computer  programs  (design  changes),  and  the 
problems  corrected  within  the  computer  program  by  the  new  release.  The 
following  relationship  was  derived: 

Number  of  Problems  * 0.4  0.00037Ng  + 0.00069Ng  x 

+ 0.00048Ng  X Np 

where  =»  number  of  source-code  Instructions 

Np  = number  of  design  changes 
Np  = number  of  problems  corrected 

The  statistics  Indicate  that  each  variable  In  the  above  equation  Is  sig- 
nificant in  explaining  the  reported  problems  with  the  software  version 

2 

once  It  was  released  for  operational  use.  The  R (0.64)  Is  marginal, 
but  promising. 

Finally,  the  choice  ot  programming  language  should  have  a dramatic 
Impact  on  maintenance.  The  goal  of  high-level  programming  languages  is 
to  make  programs  more  readable  and  easily  modified;  therefore,  programs 
in  a high-level  language  should  exhibit  lower  maintenance  costs.  The 
ADPREP  data^  contains  information  that  allows  this  hypothesis  to  be  tested. 
For  each  program  reported  on,  the  staffing  of  the  maintenance  function 
is  described  In  terms  of  the  total  '.umber  of  maintenance  people  assigned 
per  month. 

In  Fig.  3.2,  the  number  of  maintenance  personnel  is  plotted  for 
programs  written  In  assembly  language  (dots)  and  programs  written  in  a 
high-level  language  (squares).  Separate  trends  apparently  exist,  indicat- 
ing that  the  use  of  a hlgh-order  language  makes  a significant  difference 
In  the  staffing  required  for  maintenance. 
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NUMBER  OF  MAINTENANCE  PERSONNEL 
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Figure  3.2.  Maintenance  Requirements  for  Assembly-Language  and  High-Order 
Language  Programs  (Figure  6.14  of  Volume  I) 

A linear  regression  for  the  two  languages  was  performed,  with  number 

2 

of  object-code  Instructions,  M , as  the  Independent  variable.  R sta- 
tistics are  not  very  good,  and  the  relationships  are  therefore  tenuous. 

The  results  were: 

Assembly  Language  Programs 

Number  of  Staff  ■ 1.14  + 0.00019M 
R^  - 0.43 

Hlgh-Order  Language  Programs 

Number  of  Staff  - 0.34  -f  O.OOOOSN 
R^  - 0.38 
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Based  upon  this  Initial  examination  of  the  maintenance  activity, 
several  conclusions  about  the  maintenance  process  can  be  developed.  The 
first  is  that  the  program- Improvement  aspect  of  maintenance  is  a regular, 
repeating  process  for  computer  programs  that  have  a long  and  useful  life- 
time. This  aspect  of  maintenance  consumes  a significant  portion  (if  not 
the  majority)  of  the  software  maintenance  resources.  How  maintenance 
staffing  is  determined  is  not  clear,  but  the  extent  to  which  programs 
are  improved  is  probably  based  on  the  residual  manpower  available  after 
error-correction  activities  for  the  currently  operational  version  of  the 
computer  program. 

Second,  the  error  correction  aspect  of  program  maintenance  appears 
to  follow  some  general  trends.  The  total  number  of  problems  one  expects 
to  encounter  is  probably  limited,  with  the  limit  related  to  program  size, 
the  development  history  and  the  number  of  modifications  installed  in  the 
program  during  maintenance.  Furthermore,  the  mean  time  between  software 
error  detection/correction  will  increase  with  time  and  corresponding  error 
correction.  Thus,  for  a static  program  (without  program  Improvements), 
one  should  expect  to  be  able  to  taper  the  maintenance  effort  with  time. 

Third,  the  decision  rule  used  to  determine  staffing  levels  is  not 
clear  although  fewer  resources  are  apparently  required  to  maintain  higher 
order  language  programs  than  assembly  language  programs.  Maintenance 
staffing  levels  have  apparently  not  been  determined  solely  by  estimates  of  anti- 
cipated error  correction  (true  maintenance).  Resources  are  included  for  product 
Improvement  (new  development),  although  how  and  to  what  extent  these 
determine  staffing  level  is  not  clear. 
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